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Distributed  Information  Fusion  via  Control  in  the  Space  of  Probability  Measures 

Among  other  related  directions,  the  primary  topic  of  this  project  was  to  consider  distributed 
consensus  and  information  fusion  in  the  space  of  probability  measures.  In  particular: 

Distributed  consensus  in  the  Wasserstein  metric  space  of  probability  measures  was  introduced 
under  this  project.  Convergence  of  each  agent's  (or  node's)  measure  to  a  common  probability 
measure  is  proven  under  a  weak  network  connectivity  condition.  The  common  measure  reached  at 
each  agent  is  one  minimizing  a  weighted  sum  of  its  Wasserstein  distance  to  all  initial  agent 
measures.  This  measure  is  known  as  the  Wasserstein  barycenter.  Special  cases  involving  Gaussian 
measures,  empirical  measures,  and  time-invariant  network  topologies  are  considered,  where 
convergence  rates  and  average-consensus  results  are  given.  This  algorithm  has  potential 
applicability  in  computer  vision,  machine  learning  and  distributed  estimation,  etc. 

This  work  and  its  applications  are  found  in  the  following  papers  and  technical  articles: 

•  Adrian  N.  Bishop  and  Arnaud  Doucet,  "Consensus  in  the  Wasserstein  Metric  Space  of 
Probability  Measures",  2015,  arxiv.org/abs/1404.0145 

•  Adrian  N.  Bishop  and  Arnaud  Doucet,  "Distributed  Nonlinear  Consensus  in  the  Space  of 
Probability  Measures",  In  Proceedings  of  the  19th  International  Federation  of  Automatic 
Control  World  Congress,  August,  2014,  Cape  Town,  South  Africa. 

These  papers  are  attached  in  the  appendices1. 

Focus  of  the  above  Wasserstein  (barycenter)  consensus  algorithm  to  the  problem  of  distributed 
combining  (randomly)  sampled  probability  measures  (so  called  empirical  measures)  was  explored  in 
the  following  articles  (attached  in  the  appendices): 

•  Thibaut  Lienart,  Yee  Whye  The  and  Arnaud  Doucet,  "Expectation  Particle  Belief  Propagation", 
In  Proceedings  of  the  Advances  in  Neural  Information  Processing  Systems  28  (NIPS), 
December,  2015,  Montreal,  Canada. 

•  Adrian  N.  Bishop,  "Information  fusion  via  the  Wasserstein  barycenter  in  the  space  of 
probability  measures:  Direct  fusion  of  empirical  measures  and  Gaussian  fusion  with  unknown 
correlation".  In  Proceedings  of  the  17th  International  Conference  on  Information  Fusion 
(FUSION),  July,  2014,  Salamanca,  Spain. 

•  Isaac  L.  Manuel  and  Adrian  N.  Bishop,  "Distributed  Monte  Carlo  Information  Fusion  and 
Distributed  Particle  Filtering",  In  Proceedings  of  the  19th  International  Federation  of 
Automatic  Control  World  Congress  (IFAC  WC),  August,  2014,  Cape  Town,  South  Africa. 

•  Adrian  N.  Bishop,  "Gossip-based  distributed  data  fusion  of  empirical  probability  measures".  In 
Proceedings  of  the  2014  IEEE  Workshop  on  Statistical  Signal  Processing  (SSP),  June,  2014, 
Brisbane,  Australia. 

Research,  development  and  collaborative  work,  on  the  fusion  of  empirical  measures  and  on  fusion 
methods  that  are  robust  to  unknown  correlations  and  double  counting  of  information,  is  ongoing 
through  collaboration  and  partnerships  with  researchers  at  the  Air  Force  Research  Lab  (AFRL).  This 
ongoing  work  is  inspired  by,  and  extends  some  of  the  work  noted  above. 


1AII  papers  referenced  in  this  report  acknowledge  the  support  of  the  US  Air  Force  Office  of  Scientific  Research  (AFOSR) 
through  the  Asian  Office  of  Aeronautic  Research  and  Development  (AOARD). 
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A  number  of  other  topics  were  investigated  during  this  project  that  may  be  considered  related  but 
also  tangent  to  the  primary  topic  (of  Wasserstein  barycenters  and  consensus  in  the  space  of 
probability  measures).  In  particular,  focus  on  empirical  estimation  (e.g.  Monte  Carlo  methods)  for 
control  and  estimation  (including  in  the  computation  of  Wasserstein  barycenters  and  consensus) 
was  a  common  thread  throughout  this  project. 

In  the  following  two  papers,  the  focus  was  on  methods  for  computing  the  first  and  second 
derivatives  of  optimal  filters  (e.g.  on  approximating  the  corresponding  information  matrix): 

•  Pierre  Del  Moral,  Arnaud  Doucet  and  Sumeetpal  S.  Singh,  "Uniform  Stability  of  a  Particle 
Approximation  of  the  Optimal  Filter  Derivative",  SIAM  Journal  on  Control  and  Optimization, 
2015,  Vol  53,  No  3,  pages:  1278-1304,  dx.doi.org/10. 1137/140993703 

•  Arnaud  Doucet,  Pierre  E.  Jacob  and  Sylvain  Rubenthaler,  "Derivative-Free  Estimation  of  the 
Score  Vector  and  Observed  Information  Matrix  with  Application  to  State-Space  Models", 
2015,  arxiv.org/abs/1304.5768 

These  papers  are  attached  to  the  appendices.  The  importance  of  approximating  the  relevant 
derivatives  (and  in  understanding  the  stability  of  these  approximations  rigorously)  follows  from  their 
wide-spread  use  in  parameter  estimation  and  filtering;  e.g.  in  determining  the  parameters  of  a  multi¬ 
target  model  while  simultaneously  estimating  the  target  states.  One  obvious  justification  for  this 
wide-spread  use  is  the  prevalence  of  maximum  likelihood  methods  and  their  common  reliance  on 
the  use  of  derivatives. 

The  problem  of  approximating  filtering  probabilities  in  high-dimensional  state-space  models  is 
investigated  in  the  following  two  papers  (attached  in  the  appendices): 

•  Francesco  Bertoli  and  Adrian  N.  Bishop,  "Reducing  the  Bias  in  Blocked  Particle  Filtering  for 
Fligh-Dimensional  Systems",  2014,  arxiv.org/abs/1407.0220 

•  Francesco  Bertoli  and  Adrian  N.  Bishop,  "Adaptively  Blocked  Particle  Filtering  with  Spatial 
Smoothing  in  Large-Scale  Dynamic  Random  Fields",  2014,  arxiv.org/abs/1406.0136 

The  problem  of  approximating  the  optimal  controller  in  nonlinear  stochastic  optimal  control 
problems  is  investigated  in  the  following  two  papers  (attached  in  the  appendices): 

•  Francesco  Bertoli  and  Adrian  N.  Bishop,  "Nonlinear  Stochastic  Receding  Horizon  Control: 
Stability,  Robustness  and  Monte  Carlo  Methods  for  Control  Approximation",  2015, 
arxiv.org/abs/1507. 02780 

•  Francesco  Bertoli  and  Adrian  N.  Bishop,  "Monte  Carlo  Methods  for  Controller  Approximation 
and  Stabilization  in  Nonlinear  Stochastic  Optimal  Control",  In  Proceedings  of  the  17th  IFAC 
Symposium  on  System  Identification  (SYSID),  October,  2015,  Beijing,  China. 

High-dimensional  filtering  and  control  in  complex  (e.g.  nonlinear  stochastic)  models  is  challenging  in 
practice  since  the  error  of  "most"  approximation  methods  is  somehow  exponentially  dependent  on 
model  dimension.  For  example,  naive  Monte  Carlo  integration  displays  an  error  that  is  typically 
exponential  in  the  dimension  of  the  integral.  The  filtering  and  control  papers  above  look  at 
addressing  this  dependency  by  "cutting"  up  the  model  into  components  of  lower-dimension  and 
accepting  a  bias  as  a  result.  The  total  error  may  be  subsequently  be  reduced. 
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Appendix:  Articles  and  Technical  Reports 


The  following  articles  are  now  given  in  order: 

1.  Adrian  N.  Bishop  and  Arnaud  Doucet,  "Consensus  in  the  Wasserstein  Metric  Space  of 
Probability  Measures",  2015,  arxiv.org/abs/1404.0145 

2.  Adrian  N.  Bishop  and  Arnaud  Doucet,  "Distributed  Nonlinear  Consensus  in  the  Space  of 
Probability  Measures",  In  Proceedings  of  the  19th  International  Federation  of  Automatic 
Control  World  Congress,  August,  2014,  Cape  Town,  South  Africa. 

3.  Thibaut  Lienart,  Yee  Whye  The  and  Arnaud  Doucet,  "Expectation  Particle  Belief  Propagation", 
In  Proceedings  of  the  Advances  in  Neural  Information  Processing  Systems  28  (NIPS), 
December,  2015,  Montreal,  Canada. 

4.  Adrian  N.  Bishop,  "Information  fusion  via  the  Wasserstein  barycenter  in  the  space  of 
probability  measures:  Direct  fusion  of  empirical  measures  and  Gaussian  fusion  with  unknown 
correlation",  In  Proceedings  of  the  17th  International  Conference  on  Information  Fusion 
(FUSION),  July,  2014,  Salamanca,  Spain. 

5.  Isaac  L.  Manuel  and  Adrian  N.  Bishop,  "Distributed  Monte  Carlo  Information  Fusion  and 
Distributed  Particle  Filtering",  In  Proceedings  of  the  19th  International  Federation  of 
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6.  Adrian  N.  Bishop,  "Gossip-based  distributed  data  fusion  of  empirical  probability  measures".  In 
Proceedings  of  the  2014  IEEE  Workshop  on  Statistical  Signal  Processing  (SSP),  June,  2014, 
Brisbane,  Australia. 

7.  Pierre  Del  Moral,  Arnaud  Doucet  and  Sumeetpal  S.  Singh,  "Uniform  Stability  of  a  Particle 
Approximation  of  the  Optimal  Filter  Derivative",  SIAM  Journal  on  Control  and  Optimization, 
2015,  Vol  53,  No  3,  pages:  1278-1304,  dx.doi.org/10. 1137/140993703 
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10.  Francesco  Bertoli  and  Adrian  N.  Bishop,  "Adaptively  Blocked  Particle  Filtering  with  Spatial 
Smoothing  in  Large-Scale  Dynamic  Random  Fields",  2014,  arxiv.org/abs/1406.0136 

11.  Francesco  Bertoli  and  Adrian  N.  Bishop,  "Nonlinear  Stochastic  Receding  Florizon  Control: 
Stability,  Robustness  and  Monte  Carlo  Methods  for  Control  Approximation",  2015, 
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